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PREFACE 


This  work  was  performed  under  a  task  entitled  “Development  and  Assessment  of 
ADL  Prototypes.”  This  task  is  intended  to  promote  collaboration  by  the  Services  and  by 
other  government  and  academic  partners  in  developing  technology -based  instruction.  It  is  an 
essential  component  of  the  Advanced  Distributed  Learning  (ADL)  initiative  being  under¬ 
taken  by  the  Department  of  Defense  (DoD)  in  cooperation  and  coordination  with  the  Office 
of  Science  and  Technology  Policy  (OSTP).  This  particular  document  is  intended  to  provide 
an  overview  of  technology-based  instruction. 
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EXECUTIVE  SUMMARY 


Writing  and  printed  books  have  influenced  learning  profoundly.  The  emergence  of 
writing  brought  about  a  revolution  in  learning  by  freeing  learners  from  requirements  for 
face-to-face  interaction  with  sages  and  other  human  sources  of  knowledge,  lore,  and 
information.  Although  learning  became  neither  inexpensive  nor  universally  available  after 
the  development  of  written  language,  its  content  became  available  anytime,  anywhere. 

A  second  revolution  occurred  with  the  development  of  moveable  type  and  books. 
These  developments  also  made  the  content  of  learning,  teaching,  and  educational  materials 
available  anytime,  anywhere — ^but  more  widely  and,  significantly,  less  expensively. 

Can  we  improve  on  books?  Computer  technology  arises  as  a  possibility.  Computers 
can  adapt  the  sequence  and  type  of  operations  they  perform  to  the  conditions  of  the 
moment.  In  instruction,  they  can  adapt  the  content,  sequence,  type,  difficulty,  granularity,  and 
so  forth  of  their  presentations  to  learners  or  problem  solvers  based  on  ongoing,  dynamic 
assessments  of  their  individual  needs. 

For  this  reason,  computer  technology  may  be  effecting  a  third  revolution  in  learning. 
While  preserving  the  capabilities  of  books  to  present  the  content  of  instruction  anytime, 
anywhere,  computers  can  also  provide  the  interactions  of  teachers,  instructors,  tutors,  and 
mentors  as  needed  by  individual  learners  and  users.  This  is  not  something  books,  movies, 
television,  or  videotape  technologies  can  do  affordably  or  to  any  significant  degree.  By 
presenting  the  content  and  the  interactions  of  instruction,  computers  promise  a  new  and 
significant  capability  for  learning  that  is  available  anytime,  anywhere.  This  capability  may 
produce  a  third  revolution  in  learning,  with  a  significance  equal  to  that  of  the  development  of 
writing  and  books. 

Some  findings  that  suggest  this  revolution  is  likely  and  within  reach  are: 

•  Individualization.  Experimental  studies  comparing  human  tutors  using  only 
rudimentary  techniques  of  individualization  with  classroom  instruction  have 
found  effect  sizes  as  large  as  2.00  standard  deviations  favoring  the  tutored  stu¬ 
dents.  Technology  makes  such  tutoring  capabilities  affordable. 

•  Adaptation  and  interactivity.  As  a  form  of  individualization,  branched 
instruction  that  adjusts  its  content  and  pace  for  individual  learners  has  been 
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compared  with  a  strictly  linear  presentation  of  the  same  instructional  material 
using  identical  delivery  devices.  Studies  found  effect  sizes  of  0.72  and  2.16 
standard  deviations  favoring  the  branched  approaches  and  indicate  the  value  of 
adaptation  and  interactivity  in  instruction. 

Intensity  of  instruction.  An  individual  student  in  classroom  instruction  is 
required  to  answer  an  average  of  about  0.11  questions  an  hour.  In  tutored 
instruction,  the  number  of  questions  an  individual  student  is  required  to  answer 
has  been  found  to  range  from  117  to  146  questions  an  hour.  Results  from 
some  technology-based  instruction  have  been  found  to  average  about 
120  questions  during  12-minute  sessions.  The  intensity  of  individual  tutoring 
and  technology-based  instruction  far  exceeds  that  of  classroom  instruction. 

Pace  of  instruction.  The  ratio  of  time  the  fastest  students  in  a  typical  class¬ 
room  need  to  achieve  mastery  of  material  being  presented  compared  with  the 
time  needed  by  the  slowest  students  is  about  1:5.  This  spread  of  capabilities  is 
very  difficult  for  human  classroom  teachers  to  satisfy,  despite  their  heroic 
efforts  to  do  so.  Some  students  will  inevitably  be  bored,  and  others  will  be  lost. 
Many  studies  have  found  that  allowing  students  to  use  technology-based 
instruction  to  progress  at  their  own  rates  and  receive  material  tailored  to  their 
own  needs  reduces  overall  time-to-mastery  by  about  30  percent. 

Cost-effectiveness.  When  compared  with  reduced  class  size,  increased 
instructional  time,  peer  tutoring,  and  professional  tutoring,  technology-based 
instruction  was  found  to  be  the  most  cost-effective  approach  among  these 
alternatives. 

Intelligent  tutoring  systems.  Intelligent  tutoring  systems  intended  to  replicate 
the  practices  of  expert  human  tutors  have  demonstrated  effect  sizes  of  0.84  to 
1.00  standard  deviations  when  compared  with  classroom  instruction.  These 
systems  appear  to  have  raised  significantly  the  ceiling  for  instructional 
improvements  potentially  available  from  technology-based  instruction.  They 
may  end  up  using  approaches  that  are  different  and  far  more  powerful  than 
those  used  by  human  tutors.  Through  the  use  of  web-accessed  instructional 
objects  and  personal  (perhaps  wearable)  learning  associates,  these  systems  may 
yield  unexpected  and  unanticipated  results. 

seems  reasonable  to  conclude  that  this  third  revolution  will: 

Make  the  functionalities  of  individuahzed  tutoring  widely  accessible  and 
affordable 

Permit  interactive,  individuahzed  learning  to  take  place  anytime,  anywhere 

Eventually  bring  about  profound  changes  in  our  educational  institutions  and  in 
the  roles  and  responsibilities  of  the  people  (instructors,  students,  and  adminis¬ 
trators)  who  work  in  them 


Help  bring  into  being  a  nation  of  life-long  learners  who  are  well  prepared  to 
meet  the  challenges  of  technological  change  and  thrive  in  the  global  market¬ 
place 

Produce  radical  change  in  the  practice  and  processes  of  instruction 
Change  what  we  do  and  what  we  aspire  to  do  in  instruction. 


TECHNOLOGY,  THE  COLUMBUS  EEEECT,  AND 
THE  THIRD  REVOLUTION  IN  LEARNING 


A.  INTRODUCTION 

The  apocryphal  tale  goes  something  like  this: 

An  Enlightened  Leadership  discovers  that  relatively  minor  investments  in 
educational  technology  can  significantly  enhance  the  capabilities — the  pro¬ 
ductivity,  competitiveness,  and  competence — of  their  domain.  The  next  step, 
of  course,  is  to  appoint  a  “Blue-Ribbon”  Committee.  The  task  of  the 
Committee  is  to  design  the  ideal  technology  for  education.  The  Committee 
meets,  deliberates,  and  issues  specifications  for  the  new  technology. 

Physically,  it  must  be  mgged,  lightweight,  and  easily  portable — available 
anytime,  anywhere.  It  must  operate  indoors  and  out,  under  a  wide  range  of 
temperature,  humidity,  and  other  environmental  conditions,  and  it  must 
require  only  minimal,  if  any,  external  power  support.  Functionally,  it  must 
provide  easy,  rapid,  and  random  access  to  high-quality  text,  black  and  white 
or  full-color  graphics,  and  high-resolution  photographs.  It  must  include  an 
interface  that  is  easily  understood  and  usable  by  all,  preferably  communi¬ 
cated  in  natural  language.  It  should  allow  self-pacing  (i.e.,  learners  should  be 
able  to  proceed  through  instructional  content  as  rapidly  or  as  slowly  as 
needed).  It  should  be  suitable  for  life-long  learning  and  readily  available  to  a 
wide  range  of  users  in  home,  school,  and  workplace  settings.  Economically, 
it  must  be  inexpensive,  or  as  the  Committee  reports  “require  only  minimal 
financial  investment  on  the  part  of  potential  end  users.” 

The  Enlightened  Leadership  receives  the  Committee’s  report  with  relief. 
Development  of  the  technology  will  require  no  lengthy  research  and  devel¬ 
opment,  no  new  taxes,  no  new  infrastructure,  and  no  difficult  political  or 
administrative  decisions  or  compromises.  In  fact,  the  only  thing  it  will 
require  is  business  as  usual.  The  reason  may  be  as  obvious  to  the  readers  as 
it  is  to  the  Enlightened  Leadership.  The  recommended  technology  is,  of 
course,  the  technology  of  books — which  are  already  available  and  in  place. 

Writing  and  books  effected  revolutions  in  learning  or,  more  precisely,  how  we  go 
about  the  business  of  learning.  Before  their  appearance,  during  the  earlier  100,000  or  so 
years  of  human  (i.e.,  homo  sapiens)  existence,  instruction  took  place  person  to  person.  It 
was  expensive  and  slow  and  produced  uneven  results  because  it  depended  heavily  on  the 
teacher’s  knowledge,  capabilities,  and  instructional  expertise.  Matters  concerning  more  than 
basic  subsistence  reached  very  few  people. 
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Writing  developed  about  7,000  years  ago  and  progressed  from  picture-based  ideo¬ 
graphs  to  consonants  and  vowels  represented  with  alphabet-based  phoneticization  by  per¬ 
haps  1000  BC.  It  allowed  the  content  of  advanced  ideas  and  teaching  to  transcend  time  and 
place.  Because  of  that  capability,  it  effected  a  major  revolution  in  learning.  People  with 
enough  time  and  resources  could  study  the  words  of  the  sages  who  went  before  them  with¬ 
out  having  to  rely  on  face-to-face  interaction  or  the  vagaries  of  human  memory. 

As  discussed  by  Kilgour  (1998),  books  (i.e.,  something  beyond  mud  or  stone  tab¬ 
lets)  were  copied  first  on  papyrus  and  later  on  parchment  rolls  until  about  300  BC  when  the 
Romans  began  to  sew  sheets  of  parchment  together  into  codices.  These  codices  resembled 
the  books  of  today  and  allowed  easier  and  random  access  to  the  content.  They  were  also 
less  costly  to  produce  than  papyrus  rolls  because  they  were  based  on  locally  available 
parchment  made  from  animal  skin  and  allowed  content  to  be  placed  on  both  sides  of  the 
sheets.  Use  of  paper  prepared  from  linen  and  cotton  [about  100  AD  (China)  and  1200  AD 
(Europe)]  made  books  even  less  expensive.  Still,  books  were  by  no  means  inexpensive. 
However,  their  lowered  costs  made  them  more  available  to  a  literate  and  growing  middle 
class  who  increased  the  demand  for  books  and  the  learning  they  provided.  This  increased 
demand,  in  turn,  lowered  costs  even  further. 

The  full  technology  called  for  by  the  Committee  mentioned  previously  finally 
became  available  with  the  introduction  of  books  printed  from  moveable  type  (Kilgour, 
1998).  These  printed  books  were  first  produced  in  China  around  1000  AD  and  in  Europe  in 
the  mid  1400s.  At  this  point,  the  content  of  knowledge  and  teaching  became  widely  and 
increasingly  inexpensively  available.  The  only  item  lacking  from  the  Committee’s  list  of 
specifications  was  the  availabihty  of  high-resolution  photographs,  which  had  to  await  the 
development  of  photography  in  the  mid  1800s. 

Writing  and  printed  books  have  impacted  learning  profoundly.  It  seems  reasonable 
to  view  the  emergence  of  writing  as  (among  other  things)  the  first  major  revolution  in 
learning.  Although  learning — the  acquisition  of  knowledge — was  still  neither  inexpensive 
nor  widely  available,  it  no  longer  required  face-to-face  interaction  with  sages.  By  making  the 
content  of  learning,  teaching,  and  educational  material  widely  and  inexpensively  available 
anytime  and  anywhere,  the  development  of  books  printed  from  moveable  type  effected  a 
second  revolution  in  learning. 
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B.  THE  THIRD  REVOLUTION  IN  LEARNING 


At  this  point,  we  might  consider  an  argument  often  advanced  but  best  articulated  by 
Clark  (1983).  In  discussing  all  the  means  (media)  we  have  for  delivering  instruction,  Clark 
asserted,  “the  best  current  evidence  is  that  media  are  mere  vehicles  that  dehver  instruction 
but  do  not  influence  student  achievement  any  more  than  the  truck  that  delivers  our  groceries 
causes  changes  in  our  nutrition”  (p.  445).  Of  course,  Clark  had  in  mind  more  recently 
developed  media  than  books,  but  printed  books  seem  fair  game  for  discussion  in  this 
context. 

Are  books  “mere  vehicles”  for  delivering  instruction?  Do  they  deserve  more  credit 
than  that  for  influencing  student  achievement?  Just  as  trucks  are  essential  components  in  an 
infrastructure  that  has  improved  the  nutrition  of  nations,  books  are  essential  components  in 
an  infrastructure  that  has  improved  the  learning,  performance,  and  competence  of  people 
everywhere.  Both  trucks  and  books  may  be  vehicles,  but  their  contributions  seem  to  be  more 
fundamental  than  “mere.” 

However,  the  heart  of  Clark’s  argument  remains  sound.  Books  do  not  guarantee 
learning  or  student  achievement.  Ignorance  remains  plentiful,  even  though  books  have 
diminished  its  supply  appreciably.  Clark’s  concern  can  be  summed  up  by  the  notion  that 
technology  alone  does  not  define  an  instructional  approach.  What  matters  is  what  is  done 
with  the  technology.  This  point  of  view  seems  fair  and  unequivocal.  The  presence  of  any 
technology  does  not  guarantee  that  effective  instructional  content,  effective  ways  to  present 
this  content,  or  even  the  unique  strengths  of  the  technology  itself  will  be  present  or  used.  On 
the  other  hand,  the  absence  of  a  technology  is  a  reasonable  guarantee  that  its  functionalities 
will  be  missing.  Without  printed  books,  we  may  be  back  to  the  1400s. 

How  do  we  improve  on  books?  Do  we  have  anything  better?  Computer  technology 
arises  as  a  possibility.  One  of  the  most  important  statements  in  higher  order  computer  lan¬ 
guages  (based  necessarily  on  what  is  available  in  every  digital  computer’s  lower-order 
instruction  set),  is  the  “if’  statement.  This  statement  is  of  the  (very)  general  form: 

If  <some  condition>  is  true,  then  do  <something> ;  otherwise,  do  <something  else>. 

We  can  marvel  at  the  capabilities  of  computers  to  perform  millions  of  operations  a 
second  with  perfect  accuracy  on  the  immense  amounts  of  data  that  they  retrieve.  However, 
the  central  point  for  this  discussion  is  the  capability  of  computers  to  adapt  the  sequence  and 
the  type  of  operations  they  perform  based  on  conditions  of  the  moment.  More  specifically, 
they  can  adapt  the  content,  sequence,  type,  difficulty,  granularity,  and  so  forth  of 


3 


presentations  to  learners  and  other  users  based  on  their  assessment  of  learners’  and  users’ 
current  needs. 

For  this  reason,  computer  technology  may  effect  a  third  revolution  in  learning. 
While  preserving  the  capabilities  of  writing  and  books  to  present  the  content  of  excellent 
instruction  anytime,  anywhere,  computers  can  also  provide  the  interactions  of  excellent 
teachers,  instructors,  tutors,  and  mentors  as  needed  by  individual  learners.  This  is  not 
something  books,  movies,  television,  or  videotape  technologies  can  do  affordably  or  to  any 
appreciable  degree.  This  interactivity  is  a  new  and  significant  capability.  It  is  the  core  of 
what  future  commentators  may  view  as  the  third  revolution  in  learning. 

C.  DOES  INTERACTIVITY  MATTER? 

Much  of  the  remainder  of  this  discussion  centers  on  whether  the  instructional  inter¬ 
activity  provided  by  technology  matters.  Can  we  expect  a  revolution  in  instruction  equivalent 
to  that  wrought  by  writing  and  books?  What  can  we  say  about  the  nature  of  this  revolution 
and  what  it  implies  for  instructional  practice? 

The  question  of  whether  interactivity  matters  is  addressed  to  some  degree  by  studies 
in  which  as  much  of  the  instruction  as  possible  is  held  constant  except  for  the  level  of  inter¬ 
activity.  Fowler  (1980)  and  Verano  (1987)  performed  two  such  studies.  Fowler  compared 
branched  presentations  using  computer-controlled,  adaptive  videodisc  instruction  with 
instruction  in  which  the  same  materials  were  held  to  a  fixed-content,  linear  sequence.  She 
reported  an  effect  size  of  0.72  (roughly,  an  improvement  from  the  50*  to  the  76*  percentile 
in  performance)  for  ability  to  operate  and  locate  faults  on  a  movie  projector,  which  was  the 
object  of  her  instruction.  Similarly,  Verano  also  compared  an  interactive,  adaptive,  branching 
approach  for  presenting  instructional  material  with  a  strictly  linear  approach  used  to  present 
identical  instruction  in  beginning  Spanish.  Both  of  his  treatments  used  videodisc 
presentations.  He  reported  an  effect  size  of  2.16  (roughly,  an  improvement  from  the  50*  to 
the  98*  percentile  in  performance)  in  end-of-course  knowledge.  These  two  studies,  among 
others,  suggest  that  interactivity — at  least,  interactivity  as  defined  by  these  studies — matters 
and  perhaps  a  great  deal.  Of  course,  there  is  more  to  the  story. 

D.  TUTORING  AND  THE  INDIVIDUALIZATION  OE  INSTRUCTION 

Individualized  tutoring  (one  student  working  with  one  instructor)  has  been  viewed 
and  used  for  a  long  time  as  an  effective  instructional  procedure.  Evidence  of  its  value  is 
found  in  its  continued  use  for  instruction  in  highly  complex  and  high-value  activities,  such 
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as  aircraft  piloting,  advanced  scientific  research,  and  specialized  medical  practice.  Compari¬ 
sons  of  one-on-one  tutoring  with  one-on-many  classroom  instruction  might  be  expected  to 
favor  individualized  tutoring,  and  they  do.  However,  these  comparisons  are  surprising  not 
because  of  the  direction  of  these  findings,  but  because  of  the  magnitude  of  the  differences  in 
instructional  effectiveness. 

Benjamin  Bloom’s  results  may  be  the  most  widely  noted  of  these.  Combining  the 
findings  of  three  empirical  studies  that  compared  one-on-one  tutoring  with  classroom 
instruction.  Bloom  (1984)  reported  a  general  difference  in  achievement  of  two  standard 
deviations  (roughly,  an  improvement  from  the  50*  to  the  98*  percentile  in  performance)  in 
favor  of  tutoring.  On  the  basis  of  considerable  empirical  evidence,  these  and  similar  studies 
suggest  that  differences  between  the  results  of  one-on-one  tutoring  and  classroom  instruc¬ 
tion  are  not  just  likely  but  very  large. 

Why,  then,  do  we  not  provide  these  manifest  benefits  to  all  our  students?  The 
answer  is  straightforward  and  obvious  and  has  doubtless  already  occurred  to  the  reader:  we 
cannot  afford  it.  The  issue  is  not  effectiveness,  but  cost.  Unless  our  policies  toward  educa¬ 
tional  funding  change  dramatically,  we  cannot  afford  a  single  tutor  for  every  student.  Bloom 
(1984)  popularized  this  issue  as  the  Two-Sigma^  Problem. 

In  1975,  Scriven  argued  that  individualized  instruction  was  an  instructional  impera¬ 
tive  and  an  economic  impossibility.  Is  it?  Must  instruction  remain  constrained  by  this  real¬ 
ity? 

Enter  Moore’s  (famous)  Law.2  As  recounted  by  Mann  (2000),  Electronics  maga¬ 
zine  interviewed  Moore  in  1965  and  inquired  about  the  future  of  the  microchip  industry.  To 
make  a  point,  Moore  noted  that  engineers  were  doubling  the  number  of  electronic  devices 
(basically  transistors)  on  chips  every  year.  In  1975,  Moore  revised  his  prediction  to  say  that 
the  doubling  would  occur  every  2  years.  If  we  split  the  difference  and  predict  that  it  will 
occur  every  18  months,  our  expectations  fit  reality  quite  closely.  As  Mann  points  out,  the 
consequence  of  Moore’s  Law  is  that  computers  that  initially  sell  for  $3,000  will  cost  about 
half  that  in  about  18  months. 

Moore’s  Law  has  implications  for  our  learning  applications:  Computers  are 
becoming  exponentially  less  expensive,  and  the  computational  capabilities  that  we  need  to 


1  As  in  two  standard  deviations. 

2  Gordon  Moore  is  a  semiconductor  pioneer  and  co-founder  of  Intel  Corporation. 
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support  instruction  that  is  very  much  like  individualized  tutoring  are  becoming  progressively 
affordable — if  they  are  not  already.  The  issue  then  becomes  this:  How  we  should  use  this 
increasingly  affordable  computational  power  to  support  learning? 

Since  the  1960s  (e.g.,  Suppes,  1964;  Atkinson  and  Fletcher,  1972),  we  have  had 
computer-based  instruction  (CBI)  that  could  tailor  the  content,  sequence,  and  difficulty  of 
instructional  content  to  the  needs  of  individual  learners,  and  these  approaches  were  shown  to 
be  effective.  The  Stanford  beginning  reading  programs  presented  on  Model  33  teletype¬ 
writers  running  at  110  baud  (about  10  characters  per  second)  with  randomly  accessible 
digitized  audio  achieved  effect  sizes  in  excess  of  0.80  standard  deviations  (Fletcher  and 
Atkinson,  1972).  Similar  results  were  obtained  for  elementary  school  mathematics  (Suppes, 
Fletcher,  and  Zanotti,  1975).  Instructional  approaches  used  in  these  early  programs  required 
computers  that  cost  $2-3  million;  however,  today,  computers  that  cost  under  $1,000  could 
easily  present  these  same  approaches. 

These  approaches  did  not  seek  to  mimic  directly  the  interactions  that  occur  in  human 
tutorial  instruction.  Instead,  they  attempted  to  apply  results  emerging  from  empirical  studies 
of  human  cognition,  memory,  and  learning  as  discussed  by  Suppes  (1964)  and  Atkinson 
(1972).  Efforts  to  provide  tutorial  dialogue  emerged  from  approaches  that  were  initially 
described  as  intelligent  computer-assisted  instruction  (CAI)  and  later  as  intelligent  tutoring 
systems  (ITSs)  (Sleeman  and  Brown,  1982;  Woolf  and  Regian,  2000).  They  have  been 
effective,  occasionally  yielding  effect  sizes  in  excess  of  1.00  (e.g.,  Gott,  Kane,  and  Lesgold, 
1995).  These  approaches  raise  the  following  questions:  What  accounts  for  the  success  of 
one-on-one  tutorials?  Can  we  do  the  same  with  computers? 

E.  INTENSITY  OR  INTERACTIVITY  OE  INSTRUCTION 

Graesser  and  Person  (1994)  discussed  this  issue.  They  compared  instruction  using 
one-on-one  tutoring  with  classroom  practice  in  two  curriculum  areas:  research  methods  for 
college  undergraduates  and  algebra  for  7th  graders.  Tutors  for  the  research  methods  course 
were  psychology  graduate  students,  and  tutors  for  the  algebra  course  were  high  school  stu¬ 
dents. 

Graesser  and  Person  found  the  following: 

•  Average  number  of  questions  asked  by  teachers  in  a  class  in  a  classroom  hour: 
3 

•  Average  number  of  questions  asked  by  any  one  student  during  a  classroom 
hour:  0. 1 1 
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•  Average  number  of  questions  asked  by  a  student  and  answered  by  a  tutor 
during  a  tutorial  hour: 

-  Research  methods :  2 1 . 1 

-  Algebra:  32.2 

•  Average  number  of  questions  asked  by  a  tutor  and  answered  by  a  student 
during  a  tutorial  hour: 

-  Research  methods:  117.2 

-  Algebra:  146.4. 

These  data  do  not  prove  hard-core  cause  and  effect,  but  they  show  great  differences  in  sheer 
interactivity  between  two  approaches  that  also  show  great  differences  in  instmctional 
effectiveness. 

Does  CBI  echo  this  level  of  interactivity?  Few  studies  report  the  number  of  ques¬ 
tions  students  answer  per  unit  of  time  when  using  technology.  However,  this  author 
(Fletcher)  found  that  K-3  students  who  receive  technology-based  beginning-reading  and 
beginning-arithmetic  instruction  on  the  previously  mentioned  1 10-baud  teletypewriters  were 
answering  8-12  questions  a  minute — questions  that  were  individually  assigned  and  whose 
answers  were  immediately  assessed. 

This  level  of  interactivity  would  extrapolate  to  480-720  such  questions  an  hour  if 
children  of  this  (or  any)  age  were  able  to  sustain  this  level  of  interaction  for  60  minutes. 
Instead,  these  children  generally  worked  with  the  computer-based  materials  in  daily 
12-minute  sessions,  which  extrapolates  to  96-144  individually  selected  and  rapidly  assessed 
questions  that  these  children  received  each  day.  As  mentioned  previously,  this  CAI  was 
producing  effect  sizes  in  excess  of  0.80  standard  deviations  in  comparisons  with  classroom 
instruction  in  mathematics  and  reading.  The  speed,  or  pace,  with  which  students  were 
allowed  to  progress  through  instructional  presentations,  as  much  as  anything  else,  may  be 
responsible  for  the  success  of  these  and  other  CAI  programs. 

Graesser,  Person,  and  Magliano  (1995)  point  out  that  the  neither  the  students  nor 
the  tutors  they  observed  were  particularly  sophisticated  in  their  use  of  questions.  Specifi¬ 
cally,  they  found  that  the  tutorial  techniques  long  advanced  by  researchers  and  scholars^ 
were  largely  absent.  About  half  of  the  questions  asked  by  the  students  and  their  tutors 


3  For  example,  shaping  and  fading  (Skinner,  1968),  scaffolding  (Ausubel,  1960;  Rogoff,  1990), 
reciprocal  instruction  (Palincsar  and  Brown,  1984),  error  diagnosis  and  repair  (Burton,  1982;  van  Lehn, 
1990),  or  advanced  motivational  approaches  (Lepper  et  ah,  1993). 
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required  simple  yes/no  responses.  The  techniques  the  tutors  used  were  far  from  sophisti¬ 
cated,  but  these  techniques,  as  the  data  tell  us,  were  effective.  Simple  approaches  that  aim 
primarily  to  increase  interactivity  may  by  themselves  fill  much  of  Bloom’s  two-sigma  gap. 

However,  greater  sophistication  in  one-on-one  tutoring  also  pays  off.  Semb  et  al. 
(2000)  reviewed  several  empirical  studies  of  on-the-job  training  (OJT)  and  concluded  that 
greater  knowledge  and  use  of  tutorial  techniques  result  in  greater  achievement  and  more 
efficient  learning.  These  applications  are  found  primarily  in  the  military  and  industrial 
world,  but  they  are,  effectively,  one-on-one  tutoring.  Including  advanced  tutorial  techniques 
in  our  computer-based  tutors  may  allow  us  to  exceed  Bloom’s  two-sigma  threshold.  We 
may  have  just  begun. 

F.  PACE  OF  INSTRUCTION 

The  possibility  that  simple  approaches  may,  by  themselves,  do  much  to  fill  Bloom’s 
two-sigma  gap  is  supported  by  considerations  of  pace:  the  speed  with  which  students  learn 
material  and  reach  instructional  objectives.  Easily  adjusted  pacing  is  a  capability  claimed  by 
even  the  most  rudimentary  CBI  systems. 

Many  teachers  have  been  struck  by  the  differences  in  the  pace  with  which  their  stu¬ 
dents  can  learn.  Consider,  for  instance,  some  findings  on  the  time  necessary  for  different 
students  to  reach  the  same  instructional  objectives: 

•  Ratio  of  time  needed  by  individual  kindergarten  students  to  build  words  from 
letters:  13  to  1  (Suppes,  1964) 

•  Ratio  of  time  needed  by  individual  hearing-impaired  and  Native  American  stu¬ 
dents  to  reach  mathematics  objectives:  4  to  1  (Suppes,  Fletcher,  and  Zanotti, 
1975) 

•  Overall  ratio  of  time  needed  by  individual  students  to  learn  in  grades  K-8: 
5  to  1  (Gettinger,  1984) 

•  Ratio  of  time  needed  by  undergraduates  in  a  major  research  university  to  learn 
features  of  the  LISt  Processor  (LISP)^  programming  language:  7  to  1  (Private 
communication,  Corbett,  1998) 

As  with  the  differences  between  one-on-one  tutoring  and  classroom  instruction,  we 
may  not  be  surprised  to  discover  differences  among  students  in  the  speed  with  which  they 


^  LISP  is  a  language  used  for  artificial  intelligence  (AI).  AI  pioneer  John  McCarthy  formulated  LISP  in 
the  late  1950s. 
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are  prepared  to  learn.  However,  the  magnitudes  of  the  differences  may  be  much  larger  than 
we  expect.  As  we  might  expect  from  Gettinger’s  1984  review,  a  typical  K-8  classroom  will 
have  students  who  are  prepared  to  learn  in  one  day  what  it  will  take  other  students  in  the 
same  classroom  5  days  to  learn.  More  homogeneous  grouping  of  students  based  on  their 
abilities  does  not  seem  to  mitigate  this  difference.  The  students  in  Corbett’s  1998  university 
study  are  highly  selected,  averaging  well  above  1300  on  their  Standard  Acceptance  Tests 
(SATs);  however,  the  differences  in  time  they  required  to  learn  the  fundamentals  of  a 
modestly  exotic  programming  language  remain  large. 

Initially,  students’  ability  may  cause  the  differences  in  the  speed  with  which  they 
reach  given  objectives,  but  prior  knowledge  quickly  overtakes  this  effect  as  a  determinant  of 
pace  (Tobias,  1982).  Despite  our  efforts  to  sustain  common  levels  of  prior  knowledge  in 
classrooms  by  bringing  every  student  to  some  minimal  threshold  of  learning,  we  instead 
appear  to  increase  the  differences  among  students  by  about  1  year  for  every  year  they  spend 
in  elementary  school  (Heuston,  1997).  For  instance,  the  average  spread  of  academic 
achievement  in  grade  three  is  about  3  years.  By  grade  six,  it  increases  to  about  6  years.  We 
are,  then,  working  hard  to  make  the  classroom  teacher’s  job  more  difficult. 

The  challenge  that  this  diversity  presents  to  classroom  teachers  is  daunting.  How 
can  they  ensure  that  every  student  has  enough  time  to  reach  given  instructional  objectives? 
At  the  same  time,  how  can  they  allow  students  who  are  ready  to  surge  ahead  to  do  so?  The 
answer,  of  course,  despite  heroic  efforts  to  the  contrary,  is  that  they  cannot.  Most  class¬ 
rooms  contain  many  students  who  at  one  end  of  the  spectrum  are  bored  and  at  the  other  end 
are  overwhelmed  and  lost. 

One-on-one  tutoring  allows  us  to  alleviate  this  difficulty  by  adjusting  the  pace  of 
instruction  to  the  needs  and  abilities  of  individual  students.  We  can  proceed  as  rapidly  or  as 
slowly  as  needed.  We  can  skip  what  individual  students  have  mastered  and  concentrate  on 
what  they  have  not  mastered. 

As  with  intensity  or  interactivity,  we  cannot  make  a  direct  cause-and-effect  case  for 
the  contributions  of  individualized  pace  of  instruction.  However,  with  pace,  as  with  inter¬ 
activity,  we  find  that  a  large  difference  in  instructional  treatment  is  associated  with  a  large 
difference  in  instructional  outcome.  It  does  not  seem  unreasonable  to  conclude  that  the  abil¬ 
ity  to  adjust  the  pace  of  instruction  may  also  account  for  some  of  the  large  differences  that 
favor  individual  tutoring  over  classroom  instruction. 
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Again,  we  might  ask  if  CBI,  like  one-on-one  tutoring,  allows  us  to  individualize  the 
pace  of  instruction  (with  the  pace  defined  by  the  amount  of  time  it  takes  students  to  reach 
given  instructional  objectives).  Research  findings  suggest  that  it  does.  If  students  who  could 
move  through  instructional  material  more  quickly  are  prevented  from  doing  so  in 
classrooms  but  are  allowed  to  do  so  in  CBI  settings,  we  should  find  students  reaching 
instructional  objectives  more  quickly  in  CBI  settings  than  in  classroom  settings. 

This  finding  arises  repeatedly  in  reviews  of  instructional  technology.  Orlansky  and 
String  (1977)  found  that  reductions  in  time  to  reach  instructional  objectives  averaged  about 
54  percent  across  12  CBI  evaluations  used  in  mihtary  training.  Fletcher  (1991)  found  an 
average  time  reduction  of  3 1  percent  in  6  studies  of  interactive  videodisc  (IVD)  instruction 
applied  in  higher  education.  Kulik  (1994)  and  his  colleagues  found  time  reductions  of 
34  percent  in  17  studies  of  CBI  used  in  higher  education  and  24  percent  in  15  studies  of 
CBI  used  in  adult  education.  These  reviews  are  effectively  independent  because  they 
reviewed  different  sets  of  evaluation  studies.  From  these  reviews,  it  seems  reasonable  to 
expect  time  reductions  of  about  30  percent  when  students  use  CBI  to  reach  a  variety  of 
given  instmctional  objectives. 

It  is  not  certain  that  these  reductions  result  from  the  speed  with  which  students  pro¬ 
gress  through  fixed  sets  of  items,  from  adjustments  in  content  to  take  advantage  of  what 
students  already  know  or  have  mastered,  or  from  some  combination  of  these.  However,  if 
we  simply  consider  pace  to  be  the  rate  with  which  students  reach  instructional  objectives,  it 
seems  reasonable  to  conclude  that  CBI  reduces  time  to  learn,  as  does  one-on-one  tutoring, 
primarily  by  not  holding  back  students  who  are  ready  to  progress. 

There  are  three  points  to  add  to  this  discussion: 

•  First,  the  self-pacing  enabled  by  technology-based  instruction  does  not  simply 
allow  students  to  skip  through  content  as  rapidly  as  they  can.  Instead,  most 
and  perhaps  all  technology-based  instruction  includes  an  executive  agent.  This 
executive  agent  might  be  called  an  instructional  management  system.  It  allows 
students  to  progress  through  content  as  rapidly  as  possible — but  only  after 
they  demonstrate  their  readiness  to  do  so.  The  instructional  management  sys¬ 
tem  ensures  instructional  progress  and  quality  in  ways  that  books,  as  passive 
media,  cannot. 

•  Second,  reducing  instructional  time  by  30  percent  is  a  fairly  conservative  tar¬ 
get.  Commercial  enterprises  that  develop  technology-based  instruction  for  the 
Department  of  Defense  (DoD)  regularly  base  their  bids  on  the  expectation  that 
they  can  reduce  instructional  time  by  50  percent,  while  holding  instructional 
objectives  constant.  Noja  (1991)  has  reported  technology -based-instruction 
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time  savings  as  high  as  80  percent  in  training  operators  and  maintenance  tech¬ 
nicians  for  the  Italian  Air  Force. 

•  Third,  time  saved  in  learning  is  not  a  trivial  matter.  The  Department  of  Defense 
spends  about  $4  billion  a  year  on  specialized  skill  training,  which  is  the  post- 
basic  training  needed  to  qualify  people  for  the  many  technical  jobs  (e.g., 
wheeled  vehicle  mechanics,  radar  operators  and  technicians,  medical 
technicians)  needed  to  perform  military  operations.  If  DoD  could  reduce  by 
30  percent  the  time  to  train  20  percent  of  the  people  undergoing  specialized 
skill  training,  it  would  save  over  $250  million  per  year.  If  it  could  do  so  for  60 
percent  of  the  people  undergoing  specialized  skill  training,  it  would  save  over 
$700  million  per  year. 

Assigning  dollar  values  to  the  time  that  students  spend  in  educational  settings,  espe¬ 
cially  our  K-12  classrooms,  is  more  difficult.  This  difficulty  may  account  for  the  paucity  of 
results  that  can  be  found  for  time  savings  in  K-12  education.  However,  time  so  spent  or 
saved  is  not  without  cost  and  value.  Aside  from  the  obvious  motivational  issues  of  keeping 
students  interested  and  involved  in  educational  material,  students  who  use  their  time  well  will 
profit  themselves  and  any  society  that  depends  on  their  eventual  competency  and 
achievement.  The  time  savings  that  technology-based  instruction  offers  in  K-12  education 
could  be  more  significant  and  of  greater  value  than  the  time  savings  obtained  in  post-educa¬ 
tion  training. 

G.  COST  EFFECTIVENESS 

The  issue  for  any  educational  decision-maker  faced  with  a  unyielding  budget,  an 
unpredictable  revenue  stream,  and  unending  demands  for  expenditures  that  are  both  urgent 
and  imperative  is  not  limited  to  instructional  effectiveness.  The  core  of  such  decision¬ 
making  is  not  just  effectiveness  but  what  must  be  given  up  to  get  it.  Most  often  and  most 
specifically,  this  consideration  centers  on  costs  and  cost  effectiveness. 

Is  there  evidence  that  technology  applications  in  instruction  are  cost  effective? 
Despite  the  decision-makers’  uncompromising  need  for  such  evidence,  httle  evidence  exists 
to  aid  their  deliberations.  This  situation  is  especially  prevalent  among  innovations,  such  as 
technology-based  instruction,  where  researchers  often  seek  to  learn  if  an  approach  “works” 
or  if  it  works  better  than  existing  practice  but  seldom  to  determine  if  it  works  well  enough  to 
justify  its  expense.  The  expense  issue  requires  consideration  of  costs,  cost  models,  and 
similar  issues  that  researchers  in  instructional  procedures  and  practices  prefer  to  leave  to 
others.  An  innovation’s  cost  effectiveness  is  rarely  considered  by  anyone  other  than  the 
decision-maker  who  will  be  pressured  to  adopt  it. 
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Of  course,  asking  if  an  approach  is  cost  effective  oversimplifies  the  issue.  Cost 
effectiveness  is  a  relative  term.  We  cannot  meaningfully  label  some  approach  as  cost  effec¬ 
tive  without  specifying  the  alternatives  with  which  this  approach  is  being  compared.  Cost- 
effectiveness  studies  require  the  use  of  a  single  experimental  paradigm — using  comprehen¬ 
sive  models  of  costs  and  effectiveness — to  compare  the  alternatives  under  consideration. 
Typically,  cost-effectiveness  investigators  either  observe  different  levels  of  effectiveness 
achieved  while  holding  costs  constant  or  they  observe  the  different  costs  required  to  reach 
fixed  thresholds  of  effectiveness. 

Such  comparisons  in  technology-based  instruction  are  hard  to  find — even  in  indus¬ 
trial  training  where  all  decisions  are  a  matter  of  profit  and  loss  and  in  military  training  where 
allocations  of  resources  may  literally  be  a  matter  of  life  and  death.  Fletcher  (1997)  reported 
a  limited  cost-effectiveness  argument  for  technology-based  education.  He  presented 
empirical  data  gathered  from  earlier  studies  by  Jamison  et  al.  (1976),  Levin,  Glass,  and 
Meister  (1987),  and  Fletcher,  Hawley,  and  Piele  (1990)  to  compare  the  costs  (adjusted  for 
inflation)  of  different  educational  interventions  to  raise  5*  grade  mathematics  scores  on  a 
standard  achievement  test  by  one  standard  deviation.  Providing  daily  10-minute  CBI  ses¬ 
sions  was  found  to  be  less  expensive  (and,  hence,  more  cost  effective)  than  peer  tutoring, 
professional  tutoring,  decreasing  class  size  from  35  to  either  30  or  20,  or  increasing  the 
length  of  the  school  day  by  30  minutes.  This  finding  suggests  that  a  strong  cost-effective¬ 
ness  position  for  technology-based  instruction  is  likely  even  at  this  early  (relative  to  what 
may  be  coming)  stage  of  development.  Given  that  we  are  most  likely  in  the  horseless  car¬ 
riage  years  of  the  third  revolution  in  learning,  these  are  promising  results.  More  work  of  this 
sort  is  needed. 

H.  INTELLIGENT  TUTORING  SYSTEMS 

If  interactivity  and  individualization  of  pace  can  be  achieved  by  standard  approaches 
to  CBI,  is  there  any  reason  to  pursue  more  exotic  approaches?  Specifically,  is  there  any 
reason  to  develop  what  are  called  intelligent  tutoring  systems  (e.g.,  Woolf  and  Regian, 
2000)? 

“Intelligent  tutoring  systems”  may  be  as  intelligently,  or  unintelligently,  designed 
as  any  others.  They  involve  a  capability  that  has  been  developing  since  the  late  1960s  (Car- 
bonell,  1970)  but  has  only  recently  been  expanded  into  general  use.  In  this  approach,  an 
attempt  is  made  to  mimic  directly  the  one-on-one  dialogue  that  occurs  in  tutorial  interac¬ 
tions.  Carbonell  was  a  computer  scientist  who  focused  on  the  underlying  computation 
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capabilities  needed  to  support  this  approach.  He  contrasted  ad-hoc  frame-oriented  (AFO) 
approaches  with  information  structure-oriented  (ISO)  approaches.  Today,  we  might  be  more 
likely  to  discuss  “knowledge  representation”  as  the  requisite  capabihty,  but,  in  either  case, 
the  software  requirement  is  to  represent  human  knowledge — ^knowledge  of  the  subject 
matter,  knowledge  of  the  state  of  the  student,  and  knowledge  of  teaching  strategies. 

More  important  for  those  who  wish  to  focus  on  instructional  rather  than  computa¬ 
tional  capabilities  are  the  functionalities  that  distinguish  intelligent  tutoring  systems  from 
their  predecessors.  Despite  current  marketing  efforts  to  describe  any  instructional  system 
that  uses  technology  as  being  an  intelligent  instructional  system,  clear  differences  exist 
between  what  has  long  been  the  objective  of  these  systems  and  what  has  long  been  available 
in  the  state  of  the  art. 

Two  functionalities  are  critical  and  discriminating  in  an  intelhgent  instructional  sys¬ 
tem: 

•  First,  we  expect  to  find  an  ability  to  generate  computer  presentations  and 
responses  in  real  time,  on  demand,  and  as  needed  or  requested  by  learners. 

•  Second,  we  expect  to  find  an  ability  to  support  mixed  initiative  dialogue  in 
which  either  the  computer  or  the  (human)  student  can  generate,  ask,  and  answer 
open-ended  questions. 

Notably,  instructional  designers  do  not  need  to  anticipate  and  pre-store  these  interactions. 
The  motivation  for  funding  the  development  of  intelligent  instructional  systems  in  the  early 
1970s  was  not  to  apply  artificial  intelligence  (AI)  or  ISO  techniques  to  CBI  nor  was  it  to 
mimic  one-on-one  tutorial  dialogue.  Instead,  it  was  to  reduce  the  costs  of  instmctional 
materials’  preparation  by  developing  capabihties  to  generate  them  on-line,  in  real  time. 
Generative  capabilities  were  intended  to  reduce  the  time  and  resources  needed  by  the  other 
approaches  for  anticipating  and  pre-specifying  all  possible  student-computer  interactions. 

Currently,  intelligent  tutoring  systems  are  more  sophisticated  computationally  and 
functionally  than  other  more  typical  CBI  systems,  but  they  remain  expensive  to  produce. 
These  systems  can,  of  course,  adjust  the  pace,  sequence,  interactivity,  style,  difficulty,  and  so 
forth  of  instruction  to  the  needs  of  individual  learners,  just  as  other  approaches  can.  They 
can  also  make  many  of  the  adjustments  to  individual  learners  that  human  tutors  can  make. 
The  cost  to  produce  these  systems  will  decrease  as  our  techniques  to  develop  them  improve, 
but  they  may  also  be  justified  by  increases  in  learner  achievement.  For  instance,  these 
systems  show  an  increase  in  average  effect  size  to  0.84  standard  deviations  (Fletcher,  1997), 
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which  exceeds  the  average  0.42  standard  deviations  (e.g.,  Kulik,  1994)  found  for  other  CBI 
approaches. 

However,  the  main  argument  in  favor  of  these  intelligent  tutoring  systems  is  that 
they  raise  the  bar  for  the  ultimate  effectiveness  of  technology-based  instruction.  Their 
unique  generative  and  mixed  initiative  capabilities  should  eventually  allow  richer,  more 
comprehensive,  and  more  effective  interactions  to  occur  between  students  and  the  instruc¬ 
tional  system. 

If  Kurzweil  (1999)  is  correct,  we  can  expect  a  $1,000  unit  of  computing  to  equal  the 
computational  capability  of  the  human  brain  by  the  year  2019  and  exceed  it  thereafter. 
Computers  may  then  become  more  effective  in  providing  instruction  than  human  tutors  even 
if  humans  use  aU  the  techniques  Graesser  et  al.  (1995)  found  that  they  now  neglect.  We 
may  not  be  implanting  integrated  circuits  in  our  brains  as  Kurzweil  suggests  we  might  by 
2029;  however,  using  computers  to  discover  more  than  any  human  agent  can  about  the 
unique  potential  of  every  individual  and  devising  effective  and  individualized  procedures  to 
reach  this  potential  seem  to  be  appealing  and  realistic  prospects. 

Whatever  the  case,  the  extensive  tailoring  of  instruction  to  the  needs  of  individual 
students  that  can  be  obtained  by  using  generative,  intelligent  tutoring  systems  can  only  be 
expected  to  increase.  Our  current  approaches  may  be  reaching  their  limits.  Intelligent 
tutoring  systems  may  make  available  far  greater  instructional  effectiveness  and  efficiencies 
than  we  can  obtain  from  the  approaches  we  are  using  now. 

L  THE  COLUMBUS  EFFECT  IN  INSTRUCTIONAL  TECHNOLOGY 

Prognostications  aside,  technological  approaches  to  instruction  may  provide  yet 
another  example  of  what  might  be  called  the  Columbus  Effect.  As  readers  will  recall, 
Columbus  sailed  west  intending  to  find  India  (and  a  lucrative  spice  route).  Instead,  he  (re-) 
discovered  what  became  a  new  world  for  Europeans.  Such  a  result  typifies  technological 
progress.  Seeking  one  thing  based  on  famihar,  common  practice,  we  inevitably  end  up  with 
something  else — unforeseen  and  unexpected.  Wireless  telegraph  produced  something 
functionally  quite  different  from  the  telegraph — namely,  the  radio.  Similarly,  efforts  to  make 
a  carriage  move  without  a  horse  produced  automobiles,  to  say  nothing  of  gas  stations, 
motels,  and  the  Santa  Monica  Ereeway.  In  seeking  affordable  one-on-one  tutoring  through 
automation,  we  may  end  up  with  something  no  one  now  envisions.  The  metaphor  based  on 
current  practice  gets  us  started.  The  result  may  surprise  us  all. 
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As  we  begin  with  a  vision  of  one-on-one  tutoring  made  affordable  through  technol¬ 
ogy,  our  work  may  center  on  efforts  to  mimic  the  interactions  that  occur  between  human 
tutors  and  their  students.  We  may  be  pursuing  human-less  tutoring  (just  as  books,  televi¬ 
sion,  and  other  noninteractive  media  may  be  viewed  human-less  lecturing),  but  we  are  likely 
to  end  up  with  something  quite  different  in  function,  appearance,  and  use. 

The  advanced  distributed  learning  (ADL)  initiative  currently  being  pursued  jointly 
by  DoD  and  the  White  House  Office  of  Science  and  Technology  Policy  (OSTP)  gives  us  a 
hint  of  what  this  different  result  may  be.  This  vision  is  based  on  the  expectation  that  most,  if 
not  all,  human  knowledge  will  become  available  as  shareable,  interoperable  objects  on  the 
World  Wide  Web.  The  ADL  initiative  seeks  to  specify  and  develop  these  objects,  but  it 
does  so  because  it  envisions  something  that  might  be  called  a  personal  learning  associate 
(PLA). 

Physically,  a  PLA  will  be  a  computer  that  is  either  carried  or  worn.  In  keeping  with 
the  suggestions  of  Kurzweil  and  others,  it  may  even  be  implanted  to  provide  direct  brain- 
computer  interaction,  although  this  possibility  seems  more  distant  and  needs  more  review 
and  consideration.  Functionally,  a  PLA  will  provide  wireless  connection  to  the  Web  or  its 
successor  in  the  global  communication  arena.  A  student  of  any  age  will  use  it  for  learning, 
and  decision-makers  (e.g.,  electronics  technician,  military  tactician,  or  business  planner)  will 
use  it  to  help  solve  practical  problems. 

All  users  will  interact  with  the  PLA  through  spoken,  natural  language.  It  will  provide 
a  full  range  of  display  capabilities,  including  text,  graphics,  and  photographs,  as  specified  by 
the  apocryphal  Committee’s  specifications.  It  will  also  provide  animation,  digital  video  of 
some  sort  augmented  by  a  full  range  of  high-fidelity  sound,  and,  perhaps,  tactile  and  haptic 
feedback.  Olfaction  remains  under  review. 

Most  of  the  PLA’s  physical  capabilities  are  state  of  the  art.  Many  of  the  software 
capabihties  are  also  available  or,  like  shareable  courseware  objects,  soon  will  be  (Fletcher 
and  Dodds,  2000).  Its  instructional  and  decision-aiding  functionalities  remain  longer  term 
and  more  elusive.  These  functionalities  call  for  the  PLA  to  develop  and  then  use  compre¬ 
hensive,  intimate,  and  accurate  knowledge  of  the  student/user  to  identify,  collect,  and  inte¬ 
grate  shareable  instructional  objects.  This  process  will  be  accomplished  on  demand  and  in 
real  time  and  will  be  precisely  tailored  to  the  individual’s  needs,  capabilities,  interests,  and 
cognitive  style.  If  the  intention  is  to  help  solve  a  problem,  information  that  the  PLA  provides 
will  not  just  be  expert,  but  will  also  be  delivered  in  a  form  that  the  individual  can  understand. 
If  the  intention  is  to  establish  a  more  permanent  change  in  the  individual’s  cognitive  abihty 
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(i.e.,  to  bring  about  learning),  the  PLA  wiU  do  so  efficiently  and  effectively  in  ways  far 
superior  to  those  that  we  now  imagine. 

Obviously,  we  have  a  way  to  go  to  realize  this  vision.  What  form  it  eventually  takes, 
how  it  is  used,  what  infrastructure  it  engenders,  and  what  impact  it  has  on  our  lives  remain  to 
be  seen,  but  its  key  capabilities  may  arise  from  the  intelligent  instructional  systems  we  are 
now  learning  to  build.  The  Columbus  Effect  will  kick  in  sooner  or  later,  but  beginning  with 
a  guiding  metaphor  based  on  individuahzed,  tutorial  instruction  and  “mentoring”  seems  a 
good  way  to  advance.  The  goal  of  learning  to  do  something  that  is  within  our  reach  but 
outside  our  grasp  has  long  been  a  stimulus  for  human  progress. 

J.  IMPACT  ON  RESEARCH  AND  THEORY 

Reviews  by  Krendl  and  Lieberman  (1988)  and  Schater  and  Fagnano  (1999)  con¬ 
tinue  to  echo  earlier  recommendations  by  Suppes  (1964)  and  others  to  apply  advances  in 
cognitive  and  learning  theory  to  the  development  of  technology-based  instruction.  Such 
efforts  will  improve  the  quality  of  instruction  delivered.  More  importantly,  these  efforts  will 
provide  feedback  to  theories  of  cognition  and  learning  to  indicate  where  they  are  right, 
where  they  might  use  some  improvement,  and  where  they  have  left  serious  gaps  that  need  to 
be  filled.  This  is  the  traditional  interplay  of  theory  and  empirical  research  that  has  served 
other  areas  of  systematic  investigation  so  well.  Technology-based  instmction,  with  its  pre¬ 
cise  control  over  stimulus  inputs  and  equally  precise  measurement  of  response  outputs, 
should  play  a  unique  role  in  completing  the  feedback  loop  between  theory  and  empirical 
research. 

Such  feedback  will  produce  significant  advances  not  just  in  instruction,  but  also  in 
related  areas.  Instructional  applications  that  use  technology  test  the  notions — or  theo¬ 
ries — of  human  cognition,  learning,  and  instruction  embodied  in  them  in  at  least  two  ways: 

•  First,  the  ability  to  put  a  proposed  notion,  model,  or  theory  into  a  computer 
algorithm  is  a  significant  demonstration.  If  a  notion  cannot  be  captured  by  an 
algorithm,  it  may  not  be  testable.  If  it  is  not  testable,  it  is  not  worth  serious 
consideration. 

•  Second,  an  instructional  application  is  an  instantiation  that  tests  the  correctness 
of  a  notion,  model,  or  theory.  To  the  extent  that  an  application  achieves  its 
goals,  the  notion(s),  model(s),  or  theory  of  cognition,  learning,  and  instruction 
on  which  it  is  based  can  be  viewed  as  correct.  However,  tests  of  instructional 
applications  seldom  yield  simple  answers.  The  fine-grain  data  that  technol¬ 
ogy-based  instruction  makes  available  will  increase  the  richness  of  the  feedback 
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we  receive.  Few  theories  whl  be  shown  to  be  perfectly  and  thoroughly  correct. 
The  more  complete  diagnostic  information  concerning  where  they  are  correct, 
where  they  are  not  correct,  and  what  they  lack  is  critical.  Detailed  and  specific 
information  of  this  sort  produces  significant  advances  in  other  fields,  and  we 
should  expect  nothing  less  from  our  instructional  applications. 

K.  THE  ENGINEERING  OE  INSTRUCTION 

Beyond  issues  of  feedback  for  theory,  our  third  revolution  in  learning  may  effect  a 
shift  in  instruction  from  art  to  engineering.  We  tend  to  view  teaching  as  an  intensely 
“human”  activity,  something  that  at  best  can  only  be  accomplished  by  human  teachers 
interacting  with  human  students.  Master  teachers  do  exist,  and  many  of  us  have  benefited 
from  the  attentions  of  at  least  one  teacher  who,  if  not  a  master,  was  at  least  able  to  impress 
on  us  the  value,  benefits,  and  pleasure  of  scholarship.  However,  such  occasions  appear  to  be 
more  the  exception  than  the  rule.  Our  perceptions  of  teaching  as  a  warmly  experienced, 
human  activity  vary  with  much  that  we  experience  in  real  classrooms,  where  each  student  is 
one  of  many  waiting  for  that  portion  of  instruction  that  addresses  his  or  her  individual 
needs. 

To  a  great  extent,  successful  instruction  is  a  matter  of  design — the  creation  of  an 
environment  to  maximize  the  probability  that  learning  will  occur  and  that  every  student  will 
achieve  specified  instructional  objectives.  Mostly,  we  seek  and  test  for  accuracy  of  know¬ 
ledge.  However,  we  may  also  seek  other  objectives  such  as  speed  of  response,  retention, 
insight,  and  transfer  of  knowledge  as  well  as  continued  interest  in  and  respect  for  the  subject 
matter.  These  goals  may  be  compatible  to  some  extent,  but,  at  some  point,  they  diverge  and 
require  different  approaches  that  compete  with  each  other  for  classroom  resources  of  human 
energy,  funding,  and,  especially,  time. 

The  design  of  such  an  environment  may  be  viewed  as  an  art — a  highly  personal,  hit 
or  miss  affair.  To  be  fair  to  our  students  and  productive  in  our  instruction,  we  need  to 
establish  a  science  of  design  that  allows  these  many  instructional  objectives  to  be  accom¬ 
plished  by  many  different  hands.  In  short,  we  need  an  engineering  of  instruction  in  which 
specific  designs  reliably  yield  specific  instructional  outcomes. 

The  notion  of  instruction  as  engineering  may  be  unpopular.  Raymond  Fox  (1994) 
has  noted  that  “one  of  the  more  difficult  problems  in  dealing  with  improvement  in  public 
education  is  to  replace  the  notion  of  teaching  as  an  art  form  with  that  of  instruction  dehvery 
as  a  systems  science”  (p.  2).  Without  technology,  precise  engineering  to  ensure  that  every 
student  achieves  a  wide  range  of  objectives  would  be  out  of  the  question.  Different  human 
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teachers,  specifically  because  they  are  human,  must  approach  their  students  differently  and 
succeed  in  different  ways  with  different  students.  One-on-one  tutoring  can  accomplish  this 
goal  to  some  degree,  but  the  precision,  “patience,”  and  consistency  of  technology  may  sur¬ 
pass  what  we  can  accomplish  with  human  tutorials.  A  technology-based  approach  may 
provide  the  individual,  precisely  tailored  attention  that  each  student  needs  to  achieve  fully  his 
or  her  potential. 

L.  IMPACT  ON  OUR  CURRICULAR  GOALS 

In  1960,  T.F.  Gilbert  wrote: 

If  you  don’t  have  a  gadget  called  a  “teaching  machine,”  don’t  get  one. 

Don’t  buy  one;  don’t  borrow  one;  don’t  steal  one.  If  you  have  such  a 
gadget,  get  rid  of  it.  Don’t  give  it  away,  for  someone  else  might  use  it.  This 
is  the  most  practical  mle,  based  on  empirical  facts  from  considerable 
observation.  If  you  begin  with  a  device  of  any  kind,  you  will  tiy  to  develop 
the  teaching  program  to  fit  that  device  [p.478  (the  italics  are  Gilbert’s)]. 

Gilbert  may  be  right  and  wrong.  He  is  certainly  correct  in  suggesting  that  instruc¬ 
tional  designers  and  developers  who  use  a  “teaching  machine”  will  try  to  fit  the  teaching 
program  to  it.  The  new  functionalities  that  such  a  device  makes  available  motivate  its  use  in 
the  first  place.  One  can  imagine  students  long  ago  poring  over  clay  tablets  or  papyrus  rolls 
once  their  teachers  learned  how  to  design  teaching  programs  to  take  advantage  of  written 
language.  The  same  can  be  said  for  printed  books,  which,  because  they  were  more 
accessible  and  less  expensive  than  papyrus  rolls  or  codices,  allowed  teachers  different 
assumptions  in  the  development  of  their  teaching  programs.  The  same  is  doubtlessly  tme 
for  our  third  instructional  revolution,  which  applies  interactive,  computer-based  technology 
to  the  problems  and  processes  of  instruction. 

It  is  less  certain  whether  such  adaptations  (e.g.,  fitting  the  teaching  program  to  the 
“device”)  are  significant  evils  to  be  avoided  at  all  costs.  Certainly,  if  technology  causes  us 
to  remove  or  de-emphasize  essential  elements  of  our  teaching  programs,  it  will  diminish 
their  effectiveness.  However,  if  technology  is  properly  applied,  it  will  improve,  if  not 
revolutionize,  the  effectiveness  and  efficiency  of  our  teaching  programs.  Researchers, 
developers,  and  instructors — not  the  technology  itself — must  ensure  that  it  does. 

In  either  case,  the  application  of  technology  will  change  what  and  how  we  teach. 
Technology  will  raise  the  bar  for  our  curricular  aspirations.  Tutorial  simulations  will  afford 
our  students  experiences,  access  to  exotic  (i.e.,  expensive  and  unavailable)  devices,  and 
immersion  in  collaborative  problem  solving  that  we  could  not  provide  in  any  other  way. 
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Intelligent  tutoring  capabilities  will  permit  tutorial  interactions  or  simple  conversations  with 
experts  and  expertise  that  would  otherwise  be  out  of  the  question.  Because  these  interac¬ 
tions  will  be  geared  to  each  student’s  level  of  ability  and  prior  knowledge,  they  will  produce 
levels  of  understanding  that  would  otherwise  be  unattainable.  Sooner  or  later,  we  will  be 
forced  by  necessity,  public  pressure,  or  our  own  professional  integrity  to  adapt  our  teaching 
programs  to  the  new  functionalities  that  technology  makes  available.  The  most  important 
consequence  of  the  third  revolution  in  instruction  may  not  be  that  it  improves  efficiency  and 
effectiveness  of  what  we  do  now,  but  that  it  changes  what  it  is  that  we  choose  to  do. 

M.  SUMMARY 

More  discussion  needs  to  take  place  about  the  third  revolution  in  learning  wrought 
by  technology,  but  that  discussion  must  await  wiser,  better-informed  commentary.  However, 
with  modest  certainty,  a  few  statements  can  be  made.  This  revolution  will: 

•  Make  the  functionalities  of  individuahzed  tutoring  widely  accessible  and 
affordable 

•  Permit  interactive,  individuahzed  learning  to  take  place  anytime,  anywhere 

•  (Eventually)  bring  about  profound  changes  in  our  educational  institutions  and 
the  roles  and  responsibilities  of  people  (teachers,  students,  and  administrators) 
in  these  institutions 

•  Help  bring  into  being  a  nation  of  life-long  learners  who  are  well  prepared  to 
meet  the  challenges  of  the  new  century  and  thrive  in  the  global  marketplace 

•  Lead  to  capabilities,  uses,  and  functionalities  of  which  we  are  now  only  dimly,  if 
at  all,  aware  (thanks  to  the  Columbus  effect) 

•  Produce  radical  change  in  the  practice  and  processes  of  instruction. 

Basically,  people  learn  just  one  way,  and  this  way  mostly  likely  requires  growth  or 
chemical  changes  in  the  synapses.  The  third  revolution  in  learning  will  not  change  learning 
at  this  level  any  more  than  writing  or  books  did.  However,  substantially  increasing  the 
probability  that  such  fundamental  changes  will  occur  across  all  manner  of  humans  in  aU 
manner  of  environments  does  seem  to  qualify  as  a  radical  change — one  that  warrants  the 
term  “revolution.”  Because  this  revolution  will  increase  the  tempo  with  which  learning 
occurs,  it  might  be  called  a  revolution  in  learning,  the  magnitude  of  which  we  have  only  seen 
twice  before  in  human  history. 
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